Extracting photographic images from video

ABSTRACT

Generating photographs from a video. Segments of the video for which frame-to-frame background motion is less than a threshold are identified. For each of the segments, video frames in the segment are combined to generate a photograph representative of the segment.

[0001] This is a continuation-in-part of copending U.S. application Ser.No. 09/096,720 filed Jun. 11, 1998.

FIELD OF THE INVENTION

[0002] The present invention relates to the field of image processing,and more particularly to automatically extracting photographic imagesfrom a video.

BACKGROUND OF THE INVENTION

[0003] Historically, video cameras and still image cameras have beenused for different applications and have occupied different consumermarkets. Most still image cameras do not possess the image storage andrapid capture capability of video cameras and therefore are unsuitablefor capturing and recording video clips. Conversely, the videoresolution and quality produced by most video cameras are too low forproducing high quality still images. Consequently, people who wish tocapture both videos and high quality still images must usually have botha video camera and a still image camera.

SUMMARY OF THE INVENTION

[0004] A method and apparatus for generating photographs from a video isdisclosed. Segments of the video for which frame-to-frame backgroundmotion is less than a threshold are identified and, for each of thesegments, the video frames in the segment are combined to generate aphotograph representative of the segment.

[0005] Other features and advantages of the invention will be apparentfrom the accompanying drawings and from the detailed description thatfollows below.

DESCRIPTION OF THE DRAWINGS

[0006] The present invention is illustrated by way of example and notlimitation in the figures of the accompanying drawings in which likereferences indicate similar elements and in which:

[0007]FIG. 1 illustrates use of a still image generation system togenerate a set of still images from a source video;

[0008]FIG. 2 illustrates a business model for providing a still imagegeneration service according to one embodiment;

[0009]FIG. 3 illustrates a selection window presented on a display of astill image generation system according to one embodiment;

[0010]FIG. 4 illustrates a window of a computer system display in whichpages of a video album according to one embodiment are presented;

[0011]FIG. 5 illustrates a still image generator according to oneembodiment;

[0012]FIG. 6 is a flow diagram of still image construction according toone embodiment; and

[0013]FIG. 7 is a diagram of a video index displayed on a computersystem display according to one embodiment.

DETAILED DESCRIPTION

[0014] A method and apparatus for generating still images from a videois described. The individual frames of the video are analyzed toautomatically identify at least three different types of shots: stillshots, pan shots and zoom shots. When a still shot is identified,multiple video frames from the still shot are combined to create asingle high-resolution image. For a pan shot, multiple video frames arestitched together to create a high-resolution panoramic image. For azoom shot, multiple video frames are combined to produce amultiple-resolution still image. In shots that include both pan andzoom, a multiple-resolution panoramic image is generated. Because theprocessing of the input video is automatic, the video can be processedunattended and without the need to learn complicated image editingoperations.

[0015] It is contemplated that the automatic generation of still imagesfrom video may be provided as a service to video camera users. A usermay deliver a video to a still image generation service which creates aset of high quality still images for the user in return for a fee. Inthis way, videos of weddings, parties, vacations, real estate tours,insurance records, etc. may be used to generate a corresponding set ofhigh quality photographic images. The video may be physically deliveredto the still image generation service in the form of a video recordingmedium such as a disk or tape, or the video may be uploadedelectronically from an end user computer. The set of still imagesgenerated from the video may likewise be provided to the user either ona physical recording medium (including the medium on which the video wassupplied) or by transmission via a communications network. For example,the still images may be provided to the end user as a set of printedphotographs, or posted on a server computer for viewing or download bythe end-user or parties authorized by the end-user. In the case ofposting the still images on the server computer, the end-user or otherauthorized party may be allowed to select which of the printedphotographs to download, paying a fee for each selected still image.

[0016] In one embodiment, still images generated from a user-suppliedvideo may be formatted into an electronic album of photographic imagesreferred to herein as a “video album.” The video album may be deliveredon a recording medium, including the medium on which the source video 10was recorded, or posted on a user-accessible computer network. In thecase of a still image generation service, the album may be preparedautomatically, with the individual photographs being arranged based ondefault criterion such as their order of appearance in the video. Textannotations of the video may be generated automatically based on thecorresponding audio track. In the case of user processing of the sourcevideo 10, the user may index the individual photographs of the videoalbum according to a number of different type of criteria including,without limitation, order of appearance in the video, nature of the shot(e.g., still image, panoramic image, zoom image), subject matter of thephotographs, user preference and so forth. The user may also enter textannotations.

[0017] In one embodiment, a still image generation service is providedin the form of a video processing kiosk which includes a disk or tapereader into which a user may insert a video recording medium. The kioskincludes a video processing engine to identify still, pan and zoom shotsas described above and to automatically display a set of high qualitystill images to the kiosk user. The kiosk may then prompt the user toselect which of the still images the user wishes to keep. In oneembodiment, the user is given the option of printing the still imagesusing a printing mechanism within the kiosk, to upload the still imagesto a server computer from which the user may later download the stillimages, or to have the still images delivered electronically to adestination address supplied by the user (e.g., an email address). Fullvideo album services may be provided as discussed above. The user may beprompted to pay a fee for initial processing, a fee for each still imageselected, or a combination of an initial processing fee and an imageselection fee.

[0018] In another embodiment, scene cuts in a video are automaticallydetected to create a set of miniature-view keyframes and correspondingtimecodes. The miniature-view keyframes, referred to herein asthumbnails, may be presented on the display of a computer system toallow a user to select entry points into the video. For example, if thevideo has been digitized (or recorded in digital form) and is accessibleby the computer system, then the user may select a thumbnail of interestto cause the video to begin playing on the display of the computersystem starting at the point in the video at which the thumbnailappears. In this way, a navigable index of the video is established,greatly simplifying the activity of searching a video for subject matterof interest.

[0019]FIG. 1 illustrates use of a still image generation system 12 togenerate a set of still images 15 from a source video 10. The sourcevideo 10 may be supplied to the still image generation system 12 in anumber of forms, including on video recording media such as magnetictape or disk, optical disk, solid state storage and so forth.Alternatively, the source video 10 may be delivered electronically, forexample, by uploading the video via a communications network to thestill image generation system 12. The source video 10 may be recorded ina number of different formats, including without limitation standardNTSC (National Television System Code) analog video or in any number ofdigital video formats. In the case of an analog format, the source video10 is digitized by the still image generation system 12 before furtherprocessing is performed.

[0020] In a preferred embodiment, the still image generation system 12is implemented by a programmed general purpose computer system and a setof one or more media readers, such as a cassette or diskette reader. Themedia readers may be installed in the computer system or operated asstandalone devices which generate an analog or digital video feed. Aframe digitizing module (often called a “frame grabber”) may be includedin the computer system to receive and digitize an analog video signalsupplied from an external analog media reader. Alternatively, theexternal media reader may generate a digital output that can be acceptedvia a communication port of the computer system.

[0021] The set of still images 15 generated by the still imagegeneration system 12 may be output in a number of forms. For example,the still image generation system 12 may include a printing device forgenerating printed images 19. Alternatively, the set of still images 15may be recorded on a portable storage medium 21, including on unusedrecording space on the medium on which the source video 10 was supplied.Further, the set of still images 15 may be output in electronic formappropriate for direct transmission to an end-user viewing system 22(e.g., via e-mail or electronic courier) or for posting on a servercomputer 17 that can be accessed via a communications network 20 such asthe Internet or other computer network. For example, the set of stillimages 15 may be posted on a server computer accessible via the WorldWide Web (the “Web”) so that an end-user may view the posted imagesusing a client computer (e.g., viewing system 22) and select whichimages of the set 15 to download.

[0022]FIG. 2 illustrates a business model for providing a still imagegeneration service according to one embodiment. Initially, a customer 25supplies a source video 10 to a still image generation service 26. Thesource video 10 may be provided, for example, on a portable storagemedium or by electronic transmission. The still image generation service26 processes the source video 10 to generate a set of still images 15.Finally, the customer provides a fee 27 to the still image generationservice in return for the set of still images. The fee may be monetaryor a supply of information such as profile information that can beresold to advertisers or other parties interested in demographicinformation. As discussed above, the set of still images 15 may beprovided as a set of prints, a set of images recorded on a storagemedium or by electronic transmission. Also, the customer 25 may bepermitted to select a subset of the still images, paying a reduced, orper-image fee. The fee 27 may be different for the different types ofstill images depending on a number of factors such as the number offrames that have been combined to produce the still image, the overallsize of the still image (e.g., in area or storage requirements), and soforth. As discussed above, the still image service 26 may be operated asa drop-off service or as a customer-operated kiosk. In the case of thedrop-off service, the customer 25 may drop off (or electronicallytransmit) the source video 10 and receive the set of still images 15later (e.g., by pickup or by electronic transmission). In the case of akiosk, the customer 25 may insert the source video 10 into a mediareader included in the kiosk and stand by while the source video 10 isbeing processed. The customer may interact with a user-interface of thekiosk to specify processing criteria and to select which of the set ofstill images 15 to keep.

[0023] In an alternative embodiment, a still image generation system 26is implemented by a programmed general purpose computer, such as adesktop or laptop computer of a computer user. In that case, still imagegeneration software is sold to the user, for example as shrink-wrap orclick-wrap software, and installed on the user's computer system.Additional equipment, such as the above-described media reader andplayback device may be required. Thus, the still image generation system26 may be implemented in the same end-user computer system that is usedto provide the viewing system 22 of FIG. 1.

[0024]FIG. 3 illustrates a selection window 30 presented on a display 29of a still image generation system according to one embodiment. The userof the still image generation system (e.g., element 12 of FIG. 1), whomay or may not be the person who has requested the set of still images,selects from among thumbnail views of still images (32, 34, 36)presented in the selection window 30, for example, by clickingthumbnails the user wishes to keep. In one embodiment, each selectedthumbnail view of a still image (32, 34, 36) is removed from theselection window 30 to a selections list 39. The selected still images41 may be printed, transmitted or otherwise delivered to the user asthey are selected or after all selections have been made. As shown, theindividual still images 32, 34, 36 may be obtained from different typesof video shots, including pan shots 31 produced by rotation ortranslation of the video camera, zoom shots 33 produced by zooming thevideo camera in or out or both, and still shots 35 produced by keepingthe video camera stationary or by user-activation of a repetitivecapture input which causes a captured frame to be automatically copiedto a number of successive frames. The still image generation system maymake the still images available one-by-one as they are created from thesource video 10, or the entire source video 10 may be processed togenerate the complete set of still images before the set of still imagesis presented to the user of the system.

[0025]FIG. 4 illustrates a window 50 of a computer system display inwhich pages 51 of a video album 52 according to one embodiment arepresented. The video album 52 contains separately viewable pages eachcontaining one or more still images (53A, 53B, 53C) that have beengenerated by combining frames of a video. Text descriptions 54A, 54B,54C are associated with each of the images, and may be automaticallyextracted from the audio track during video processing. Further, a soundand video icons may be associated with the images in the video album.When a viewer clicks the sound icon 55, a portion of the audio trackthat corresponds to the video segment used to generate still image 53Bis played. Similarly, when a viewer clicks the video icon 56, the videois presented starting at the first frame of the video segment used togenerate still image 53A. Virtual reality players may also be associatedwith the still images presented in the video album 52. For example, apanoramic player is invoked to allow the viewer to pan about withinpanoramic image 53A when the viewer clicks the PAN button 57. Similarly,a pan and zoom player is invoked to allow the viewer to pan and zoomwithin the multiple resolution still image 53C.

[0026] Although the pages 51 of the video album 52 are shown in FIG. 4as being cascaded over one another, many alternate arrangements of pagesmay be used. For example, the pages 51 may be tiled, or individuallyselected by any number of scrolling techniques. The pages 51 may also besorted based on a number of different criteria including, but notlimited to, order of appearance in the video, nature of still image(e.g., panorama, multiple resolution still, etc.), legend text (e.g.,grouping pages containing user-specified keywords together). Similarly,the individual still images of the video album 52 may be reorganizedwithin the video album according to such criteria so that, for example,the video album is chronologically ordered or images are groupedaccording to subject matter.

[0027]FIG. 5 illustrates a still image generator 60 according to oneembodiment. The still image generator 60 includes a scene changeestimator 61, a still image constructor 67, and a background motionestimator 65.

[0028] The scene change estimator 61 compares successive frames of thesource video 10 to one another to determine when a transformation of ascene in the video frames exceeds a threshold. When applied to an entiresource video 10, the effect of the scene change estimator 61 is tosegment the sequence of frames in the source video 10 into one or moresubsequences of video frames (i.e., video segments or clips), each ofwhich exhibits a scene transformation that is less than a predeterminedthreshold. The background motion estimator 65 and still imageconstructor 67 process each video segment identified by the scene changeestimator 61 to generate a composite still image having pixel valuesdrawn from two or more of the frames in the video segment. Thus, thepredetermined threshold applied by the scene change estimator 61 definesthe incremental transformation of a scene which results in constructionof a new still image of the still image set 15.

[0029] According to one embodiment, the scene change estimator 61operates by determining a transformation vector for each pair ofadjacent video frames in the source video. Herein, a first frame isconsidered to be adjacent a second frame if the first frame immediatelyprecedes or succeeds the second frame in a temporal sequence of frames.According to one embodiment, the transformation vector includes aplurality of scalar components that each indicate a measure of change inthe scene from one video frame to the next. For example, the scalarcomponents of a transformation vector may include measures of thefollowing changes in the scene: translation, scaling, rotation, panning,tilting, skew, color changes and time elapsed.

[0030] In one implementation, the scene change estimator 61 applies aspatial low pass filter to the frames of the source video 10 beforecomputing the transformation deltas between adjacent frames. After beinglow pass filtered, the individual frames in the source video 10 containless information than before filtering so that fewer computations arerequired to determine the transformation deltas. In one implementation,transformation deltas are cleared at the beginning of a video segmentand then a transformation delta computed for each pair of adjacentframes in the video segment is added to transformation deltas computedfor preceding pairs of adjacent frames to accumulate a sum oftransformation deltas. In effect, the sum of transformation deltasrepresents a transformation between a starting video frame in a videosegment and the most recently compared video frame in the video segment.In one embodiment, the sum of transformation deltas is compared againsta predetermined transformation threshold in decision block 63 todetermine if the most recently compared video frame has caused thetransformation threshold to be exceeded. The transformation thresholdmay be a vector quantity that includes multiple scalar thresholds,including thresholds for color changes, translation, scaling, rotation,panning, tilting, skew of the scene and time elapsed. In an alternateembodiment, the transformation threshold is dynamically adjusted inorder to achieve a desired ratio of video segments to frames in thesource video 10. In another alternate embodiment, the transformationthreshold is dynamically adjusted in order to achieve a desired averagevideo segment size (i.e., a desired number of video frames per videosegment). In yet another alternate embodiment, a transformationthreshold is dynamically adjusted to achieve a desired average elapsedtime per video segment. Generally, any technique for dynamicallyadjusting the transformation threshold may be used without departingfrom the spirit and scope of the present invention.

[0031] In one embodiment, if the most recently compared video framecauses caused the transformation threshold to be exceeded, the scene isdeemed to have changed at decision block 63 and the video frame thatprecedes the most recently compared video frame is deemed to be theending frame of the video segment. Consequently, if a predeterminedtransformation threshold is used, each video segment of the source video10 is assured to have an overall transformation that is less than thetransformation threshold. If a variable transformation threshold isused, on the other hand, considerable variance in the overalltransformation delta of respective video segments may result and it maybe necessary to iteratively apply the scene change estimator to reducethe variance in the transformation deltas.

[0032]FIG. 6 is a flow diagram of still image construction according toone embodiment. As discussed above, the scene change estimatoreffectively resolves the source video 10 into a plurality of videosegments each defined by a sequence of frames. Thus, at block 81, thenext video segment (e.g., video segment) within the source video 10 isidentified (or selected). If the video segment is determined to be emptyat decision block 83 (i.e., the video segment includes no frames), thenthe end of the video has been reached and still image construction forthe source video 10 is completed. Otherwise, the number of frames in thevideo segment is compared against a threshold number in decision block85 to determine whether the segment has a sufficient number of frames toproduce a still image. The threshold number of frames may bepredetermined or adaptively determined based on the lengths of thesegments of the source video 10. Also, the user of the still imagegeneration system may specify the threshold number of frames required toproduce a still image or the user may specify a starting value that maybe adapted according to the lengths of segments of the source video 10.In this way, the user of the still image generation system may controlhow many still images are generated, setting the threshold value to ahigh number of frames to reduce the number of video segments from whichstill images are constructed and setting the threshold value to a lowernumber to increase the number of video segments from which still imagesare constructed. Alternatively, in an adaptive system, a target numberof still images may be specified so that the threshold number may beautomatically increased or decreased during processing to converge onthe target number of still images.

[0033] If the number of frames in the video segment does not exceed thethreshold number of frames, then processing of the next video segmentbegins at block 81. Otherwise, at block 87, the background motionestimator inspects the video segment indicated by the scene changeestimator to identify a dominant motion of the scene depicted in thoseframes. This dominant motion is considered to be a background motion.

[0034] There are a number of techniques that may be used to identify thebackground motion in a video segment. One technique, called featuretracking, involves identifying features in the video frames (e.g., usingedge detection techniques) and tracking the motion of the features fromone video frame to the next. Features that exhibit statisticallyaberrant motion relative to other features are considered to be dynamicobjects and are temporarily disregarded. Motions that are shared by alarge number of features (or by large features) are typically caused bychanges in the disposition of the camera used to record the video andare considered to be background motions.

[0035] Another technique for identifying background motion in a videosegment is to correlate the frames of the video segment to one anotherbased on common regions and then determine the frame to frame offset ofthose regions. The frame to frame offset can then be used to determine abackground motion for the video segment.

[0036] Still other contemplated techniques for identifying backgroundmotion in a video segment include, but are not limited to,coarse-to-fine search methods that use spatially hierarchicaldecompositions of frames in the video segment; measurements of changesin video frame histogram characteristics over time to identify scenechanges; filtering to accentuate features in the video segment that canbe used for motion identification; optical flow measurement andanalysis; pixel format conversion to alternate color representations(including grayscale) to achieve greater processing speed, greaterreliability or both; and robust estimation techniques, such asM-estimation, that eliminate elements of the video frames that do notconform to an estimated dominant motion.

[0037] Still referring to FIG. 6, the still image constructor receivesthe background motion information from the background motion estimatorin block 89 and uses the background motion information to register theframes of the video segment to one another. Registration refers tospatially correlating video frames in a manner that accounts for changescaused by background motion. By registering the video frames based onbackground motion information, regions of the frames that exhibitmotions that are different from the background motion will appear in afixed location in only a small number of the registered video frames.That is, the regions move from frame to frame relative to a staticbackground. These regions are considered to be dynamic objects.

[0038] In one embodiment, the still image constructor removes dynamicobjects from frames of the video segment to produce a processed sequenceof video frames. This technique is described in copending U.S. patentapplication Ser. No. 09/096,720 filed Jun. 11, 1998, which is herebyincorporated by reference in its entirety. At block 89, the still imageconstructor generates a still image based on the processed sequence ofvideo frames and the background motion information. Depending on thenature of the background motion, construction of the still image mayinvolve combining two or more processed video frames into a single stillimage, referred to as a composite image. In one embodiment, thecomposite image may be a panoramic image or a high resolution stillimage. A panoramic image is created by stitching two or more processedvideo frames together and can be used to represent a background scenethat has been captured by panning, tilting or translating a camera. Ahigh resolution still image is appropriate when the subject of aprocessed sequence of video frames is a relatively static backgroundscene (i.e., the disposition of the camera used to record the videosource is not significantly changed). One technique for creating highresolution still images is to analyze the processed sequence of videoframes to identify sub-pixel motions between the frames. Sub-pixelmotion is caused by slight motions of the camera and can be used tocreate a composite image that has higher resolution than any of theindividual frames captured by the camera. When multiple high resolutionstill images of the same subject are constructed, the high resolutionstill images can be composited to form a still image having regions ofvarying resolution. Such an image is referred to herein as amultiple-resolution still image. As discussed above, when amultiple-resolution still image is displayed during execution of a videoalbum application program on a computer, a user can zoom in or out ondifferent regions of the image. Similarly, a user can pan about apanoramic image. Combinations of pan and zoom are also possible.

[0039]FIG. 7 is a diagram of a video index 96 displayed on a computersystem display 50 according to one embodiment. A video presentation 95is displayed in one window of the display 50, and the video index 96 isdisplayed in a separate window. In an alternate embodiment, the videoindex 96 may be displayed in a tool bar or other location within thesame window as the video presentation. The video index 96 containsminiaturized versions (thumbnails 97A-97J) of still images generatedfrom the video presentation. For the purpose of the video index, thethreshold number of frames required to signal a still image may be setto a low value so that at least one still image is constructed per scenechange. By this arrangement, the still image generation systemautomatically detects scene cuts in the source video and generates acorresponding still image. Consequently, the video index 96 contains athumbnail for each scene cut in the source video. In a preferredembodiment, each of the thumbnails 97A-97J is time correlated to thecorresponding video segment in the source video by a timecode. Thus, ifa user selects a thumbnail of interest in the index, the timecodeassociated with the thumbnail is used to identify a frame of the videothat has a corresponding time offset from the start of the video, andthe video is played starting at that time offset. In this way, a usermay navigate the video presentation 95 by selecting thumbnails ofinterest from the video index 96. In an alternate embodiment, thethumbnails 97A-97J may be correlated to the viedo presentation bysequence numbers instead of by time codes. For example, each frame ofthe source video may be numbered, so that the number of a video framethat begins a segment of the video used to generate a still image may beassociated with a thumbnail of the still image. When the user selectsthe thumbnail of the still image (e.g., by clicking a mouse button whena cursor controlled by the mouse is positioned over the thumbnail), thesource video is played starting at the first frame of the correspondingvideo segment.

[0040]FIG. 8 is a diagram of an embodiment of a processing system 100that may be used to perform the above-described processing operations,either as an end-user machine, within a kiosk or as part of a stillimage generation service. The processing system 100 includes aprocessing unit 110, memory 120, display device 130, user-input device140, communications device 150, media reader 160, frame grabber 170 andprinting device 180, each coupled to a bus structure 105. When theprocessing system forms part of a video processing kiosk, the displaydevice 130 and the user-input device 140 may be implemented by atouch-sensitive screen or other simplified user-interface. In alternateembodiments, other devices may be used to manipulate elements displayedon the display device 130 and to allow a user to input information andselections into the processing system 100. The printing device 180 ispreferably a high quality color printer, though a black and whiteprinter may also be used. In the case of a video processing kiosk, theprinter 180 is preferably enclosed within the kiosk housing, adjacent anopening through which printed output is made available to the kioskuser.

[0041] The processing unit 110 may include one or more general purposeprocessors, one or more digital signal processors or any other devicescapable of executing a sequence of instructions. When programmed withappropriate instructions, the processing unit may be used to perform theabove-described video processing operations.

[0042] The communications device 150 may be a modem, area network cardor any other device for coupling the processing system 100 to a computernetwork. The communications device 150 may be used to generate orreceive a carrier wave modulated with a data signal, for example, fortransmitting or receiving video frames, still images or text from aserver computer on the World Wide Web or other network, or for receivingupdated program code or function-extending program code that can beexecuted by the processing unit 110 to implement various embodiments ofthe present invention.

[0043] The memory 120 may include both system memory (typically, highspeed dynamic random-access memory) and various non-volatile storagedevices such as magnetic tape, magnetic disk, optical disk, electricallyerasable programmable read only memory (EEPROM), or any othercomputer-readable medium. As shown in FIG. 8, the memory 120 may be usedto store program code 122 for performing the above-described processingoperations and image data 124. The image data 124 may include, forexample, video frames that have been obtained from media reader 160 orfrom the frame grabber, or still images resulting from combination ofvideo frames. In one embodiment, when power is applied to the processingsystem 100, operating system program code is loaded from non-volatilestorage into system memory by the processing unit 110 or another device,such as a direct memory access controller (not shown). Sequences ofinstructions comprised by the operating system are then executed byprocessing unit 110 to load other sequences of instructions fromnon-volatile storage into system memory, including sequences ofinstructions that can be executed to perform the above-described videoprocessing operations. Thus, program code that can be executed toperform the above-described video processing operations may be obtainedfrom a computer-readable medium, including the above-described carrierwave, and executed in the processing unit 110.

[0044] The media reader 160 may be a video cassette tape reader, anoptical disk reader (e.g., Digital Versatile Disk (DVD) or Compact-Disk(CD)), a magnetic disk reader or any other device capable of readingvideo data from a portable storage media. If the video stored on theportable storage media is in a digital format (as in the case of adigital video camera output, for example), the content may be processeddirectly by the processing unit 110 to generate a set of still images.If the video is stored in an analog format (e.g., NTSC video), thesignal is sampled and converted to a digital representation. Theanalog-to-digital conversion may be performed by a separate conversiondevice (not shown), by the frame grabber 170 or by the processing unit110 itself. The frame grabber 170 is used to convert an analog videosignal received from a record/playback device 190 (e.g., a videocassette recorder, DVD player, DIVX player, video camera, etc.) or fromthe media reader 160 into a digitized set of video frames. The framegrabber may obtain an analog video signal from the media reader 160 viabus 105 or via a separate transmission path indicated by dashed arrow162. The output of the frame grabber 170 may be transferred to thememory 120 for processing by the processing unit 10 or processed inplace (i.e., within a buffer of the frame grabber) by the processingunit 110.

[0045] It should be noted that the individual video processingoperations described above may also be performed by specific hardwarecomponents that contain hard-wired logic to carry out the recitedoperations or by any combination of programmed processing components andhard-wired logic. Nothing disclosed herein should be construed aslimiting the processing system or other components of a still imagegeneration system to a single embodiment wherein the recited operationsare performed by a specific combination of hardware components.

[0046] In the foregoing specification, the invention has been describedwith reference to specific exemplary embodiments thereof. It will,however, be evident that various modifications and changes may be madeto the specific exemplary embodiments without departing from the broaderspirit and scope of the invention as set forth in the appended claims.Accordingly, the specification and drawings are to be regarded in anillustrative rather than a restrictive sense.

What is claimed is:
 1. A method of generating photographs from a video,the method comprising: identifying segments of the video for whichframe-to-frame background motion is less than a threshold; and for eachof the segments, combining video frames in the segment to generate aphotograph representative of the segment.
 2. The method of claim 1further comprising: automatically detecting a scene cut in the video;and selecting at least one video frame of a segment of the video thatfollows the scene cut to be a photograph.
 3. The method of claim 1wherein combining the video frames to generate a photograph comprisesstitching images in the video frames together to generate a panoramicphotograph.
 4. The method of claim 1 wherein combining video frames togenerate a photograph comprises blending pixels from the video frames togenerate a photograph having higher resolution than any one of the videoframes.
 5. The method of claim 1 wherein combining video frames togenerate a photograph comprises blending pixels from the video frames toform a multi-resolution photograph.
 6. The method of claim 1 whereinidentifying segments of the video for which frame-to-frame backgroundmotion is less than a threshold comprises identifying a succession offrames of the video that each include a portion of an image in apreceding frame.
 7. The method of claim 6 wherein identifying thesuccession of frames of the video that each include a portion of animage in a preceding frame comprises removing a dynamic object from atleast one frame of the succession of frames before comparing the atleast one frame to a preceding frame in the succession of frames.
 8. Amethod comprising: receiving a video from a customer on amachine-readable medium; processing the video to generate a set ofphotographs in return for a fee.
 9. The method of claim 8 furthercomprising recording the set of photographs on the machine-readablemedium and returning the machine-readable medium to the customer. 10.The method of claim 8 wherein receiving a video from a customer on amachine-readable medium comprises receiving the video in a data signalpropagated over a communications network.
 11. The method of claim 8wherein receiving a video from a customer on a machine-readablecomprises receiving the video on a machine-readable diskette.
 12. Themethod of claim 8 wherein processing the video to generate a set ofphotographic images comprises: identifying segments of the video thatexhibit background motion less than a threshold; and combining videoframes in each of the segments of the video to form the set ofphotographic images.
 13. The method of claim 12 wherein combining videoframes in each of the segments of the video to form the set ofphotographic images comprises stitching together images in the videoframes of at least one of the segments of the video to form a panoramicphotograph.
 14. The method of claim 12 wherein combining video frames ineach of the segments of the video to form the set of photographic imagescomprises stitching together images in the video frames of at least oneof the segments of the video to form a photograph having higher pixelresolution than any one of the video frames.
 15. The method of claim 8further comprising posting the set of photographic images on a serverthat is accessible to the customer via a computer network.
 16. Themethod of claim 8 wherein processing the video to generate a set ofphotographs comprises printing the set of photographs.
 17. An apparatusfor generating photographs from a video, the apparatus comprising: ascene change estimator to identify segments of the video for whichframe-to-frame background motion is less than a threshold; and a stillimage constructor to combine video frames in the segment to generate aphotograph representative of the segment.
 18. An apparatus forgenerating photographs from a video, the apparatus comprising: means foridentifying segments of the video for which frame-to-frame backgroundmotion is less than a threshold; and means for combining video frames inthe segment to generate a photograph representative of the segment. 19.An article of manufacture including one or more computer-readable mediathat embody a program of instructions for generating photographs from avideo, wherein the program of instructions, when executed by aprocessing unit, causes the processing unit to: identify segments of thevideo for which frame-to-frame background motion is less than athreshold; and for each of the segments, combine video frames in thesegment to generate a photograph representative of the segment.
 20. Thearticle of claim 19 wherein the one or more computer-readable mediacomprises a portable storage medium in which at least a portion of theprogram of instructions is embodied.
 21. The article of claim 19 whereinthe one or more computer-readable media comprises a propagated datasignal in which the program of instructions is embodied.